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Abstract 

Background: Stroke can result in death and long-term disability. Fast and high-quality care can reduce the impact 
of stroke, but UK national audit data has demonstrated variability in compliance with recommended processes of 
care. Though quality improvement collaboratives (QICs) are widely used, whether a QIC could improve reliability 
of stroke care was unknown. 

Methods: Twenty-four NHS hospitals in the Northwest of England were randomly allocated to participate either 
in Stroke 90:10, a QIC based on the Breakthrough Series (BTS) model, or to a control group giving normal care. 
The QIC focused on nine processes of quality care for stroke already used in the national stroke audit. The nine 
processes were grouped into two distinct care bundles: one relating to early hours care and one relating to 
rehabilitation following stroke. Using an interrupted time series design and difference-in-difference analysis, we 
aimed to determine whether hospitals participating in the QIC improved more than the control group on 
bundle compliance. 

Results: Data were available from nine interventions (3,533 patients) and nine control hospitals (3,059 patients). 
Hospitals in the QIC showed a modest improvement from baseline in the odds of average compliance equivalent 
to a relative improvement of 10.9% (95% CI 1.3%, 20.6%) in the Early Hours Bundle and 1 1.2% (95% CI 1.4%, 21.5%) 
in the Rehabilitation Bundle. Secondary analysis suggested that some specific processes were more sensitive to 
an intervention effect. 

Conclusions: Some aspects of stroke care improved during the QIC, but the effects of the QIC were modest 
and further improvement is needed. The extent to which a BTS QIC can improve quality of stroke care remains 
uncertain. Some aspects of care may respond better to collaboratives than others. 

Trial registration: ISRCTN 13893902. 
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Background 

Quality improvement collaboratives (QICs) bring together 
multidisciplinary teams across different organizations to 
work over a period of time towards defined improvement 
goals [1]. Though QICs have enjoyed enduring popularity 
as a means of securing improvement, evidence to support 
their effectiveness is equivocal [2]. This may be because 
studies of collaboratives have often used weak designs 
that are biased in favor of positive findings, tend towards 
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measurement error, and perform poorly in ruling out 
alternative explanations for any improvement detected, 
including secular trends [3,4]. Further, collaboratives may 
tackle complex, multi-faceted interventions, some of 
which may have weak underlying evidence for their effect- 
iveness [5]. In this article, we report a cluster-randomized 
trial of a QIC based on the Breakthrough Series (BTS) 
model that sought to improve compliance with processes 
associated with high-quality stroke care in hospitals in the 
Northwest of England. 

Stroke affects up to 15 million people per year world- 
wide [6]. Of these, 17% die within one month. Survivors 
often suffer long-term disability, resulting in increased 
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health and social care costs. The impact of stroke can be 
substantially mitigated by optimal care, but data from 
the 1990s showed that UK patients had poorer outcomes 
than many European countries [7]. For the last decade, 
the English National Sentinel Audit of Stroke (hereafter 
'National Audit') has collected data biennially from hos- 
pitals in England and Wales on over two hundred mea- 
sures of stroke care [8]. This has revealed dramatic 
variation in the care provided by different organizations, 
and has identified nine evidence-based processes associ- 
ated with improved outcomes [9]. In 2008, prior to the 
commencement of this research, National Audit data 
indicated that stroke care in the Northwest of England 
was among the worst in the UK; for example, only four of 
every ten patients received a brain scan within 24 hours. 

Stroke 90:10 was a BTS collaborative that aimed to 
improve compliance to an average of 90% on each of the 
nine processes by the next National Audit (2010), up 
from the baseline average of 72% in 2008. The nine 
processes targeted by the collaborative were in effect pre- 
defined, since they had already been selected as the basis 
of performance assessment for the National Audit. How- 
ever, the aims of Stroke 90:10 went beyond simply enhan- 
cing reported performance on the national measures. The 
QIC sought to improve not just average compliance, but 
also reliability. Reliability was defined to mean reducing 
variation and delivering higher levels of consistent per- 
formance. To motivate and assess reliability, the nine key 
processes were organized into two care bundles: one for 
early hours care (Bundle 1) and one for rehabilitation 
(Bundle 2) [10]. 

We conducted a cluster randomized controlled trial 
with an interrupted time series design to determine 
whether hospitals participating in the Stroke 90:10 
collaborative improved more than non-participating con- 
trols, as assessed by compliance with the two bundles of 
care. We compared across collaborative and control hos- 
pitals the amount of improvement on bundle compliance. 

Quality improvement collaboratives 

A QIC is an organized, multifaceted approach to quality 
improvement that involves five essential features: an 
agreed topic and aim; clinical and quality improvement 
experts who provide support for improvement by acting 
as a faculty; multi-professional teams from multiple sites 
who participate in the QIC; use of an agreed model for 
improvement (for example, setting an aim, collecting 
data and testing changes); and the collaborative process 
involves a series of structured activities including face-to 
face-meetings [5]. 

As well as mastering quality improvement techniques 
and applying them in their own organizations, teams par- 
ticipating in QICs are encouraged to create a sense of com- 
mon purpose and commitment, to generate enthusiasm, 



mutual support, and shared learning [11]. Several variants 
of QIC models exist, with the Institute of Healthcare 
Improvements BTS design among the most widely used 
[1]. A BTS collaborative typically involves collaborative fac- 
ulty comprising quality improvement experts, three learn- 
ing sessions (collaborative meetings), and action periods 
where participants implement activities, including Plan- 
Do-Study Act (PDSA) cycles. QICs are often used to 
implement evidence-based processes grouped into care 
'bundles' or composites of processes. Behind the concept 
of bundles is the idea that groups of interrelated processes, 
rather than individual processes, are the appropriate target 
both for standards of practice and for analysis [12]. Bundles 
require that all patients receive all elements of the bundle 
(or have documented evidence of valid exception). There is 
also a theory that the bundle as a whole will achieve better 
results than the sum of its parts [13]. 

The stroke 90:10 QIC 

The Stroke 90:10 collaborative ran from July 2008 to 
December 2010, with participating organizations recruited 
between July and December 2008. Participating site teams 
received an extensive package of support from the pro- 
gram office, which was based at Salford Royal Hospitals 
NHS Trust. Support included executive mentoring visits; 
direct access to the Stroke 90:10 project director; an 
improvement advisor and one another via a web-based 
portal (extranet); and weekly online sharing and learning 
sessions. Teams were asked to produce monthly reports 
to reflect on their performance. The project director (au- 
thor MP) also met with each hospitals chief executive and 
the team to review progress twice. 

All hospitals participating in the collaborative were 
asked to commit to: 

1. appointing an executive lead, a physician leader, a 
site lead, and a project team comprising relevant 
leaders from clinical and ward areas; 

2. taking part in one two-day and two one-day 
learning sessions, 90 days apart, that provided 
instruction in the theory and practice of 
improvement, offered teams advice and guidance, 
and shared cumulative results; 

3. participating in ongoing collaborative activities over 
the course of the program; 

4. using 'The Model for Improvement' to implement 
changes at the point of care and test them for local 
feasibility, reliability, and evidence of improvement 
in relation to the two bundles of processes relating 
to stroke care (Table 1) [14]; 

5. collecting data on 20 randomly selected patients 
per month; 

6. submitting the data to a bespoke web-based system 
linked to the National Audit. 
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Table 1 Care bundles for stroke 90:10 

Early hours (Bundle 1) 1. Brain imaging within 24 hours of admission to hospital (CT scan) to confirm stroke type (ischaemic or haemorrhagic) 
and determine management. 

2. Delivery of aspirin or an alternative antiplatelet (for patients where an antiplatelet is clinically indicated) 
within 24 hours of admission to modulate stroke complications and improve outcomes. For shorthand, we refer 
to this as 'aspirin.' 

3. Swallow screen within 24 hours of admission, to prevent unnecessary withdrawal of nutrition, support timely 
administration or modification of aspirin/antiplatelet delivery and highlight patients who need on-going 
management of swallow safety. 

4. Weight assessment on admission, as a marker of the likelihood of repeated weighing and diligent management 
of nutrition. 

Rehabilitation (Bundle 2) 1. Physiotherapy assessment within 72 hours of admission to improve early mobilization, and increased likelihood 
of targeted goal setting. 

2. Occupational therapy assessment within 4 days of admission to support activities of daily living, memory, 
perception and cognition. 

3. Mood assessment (during the in-patient stay) to screen for altered mood and other factors, given that post-stroke 
depression is known to affect the likelihood of long-term recovery. 

4. Documented evidence of MDT goals set for rehabilitation as a marker of patient involvement in care and 
multidisciplinary team working. 

5. 50% of the patient's hospital stay on a stroke unit, defined using the National Audit criteria, given evidence 
that stroke units reduce mortality and improve patient outcomes. 



Methods 

Study design 

A cluster randomized controlled trial was used with an 
interrupted time series design to compare the hospitals 
participating in the Stroke 90:10 QIC with the non- 
participating controls. The study sought to compare the 
amount of improvement on the two evidence-based 
bundles of care (early hours and rehabilitation) used in 
the QIC. 

This study was approved by Tameside and Glossop 
Research Ethics Committee (Ref: 08/H1013/55) and was 
registered as a clinical trial with the International Stand- 
ard Randomized Controlled Trial Number Register (Ref: 
ISRCTN13893902). 

Setting and participants 

All NHS hospital Trusts in the Northwest of England 
were invited to participate based on the pre-defined in- 
clusion criteria of: a minimum of ten inpatient dedicated 
stroke beds (a stroke unit'); agreement to participate 
signed by the chief executive; agreement to participate 
from a consultant in stroke medicine (or equivalent); a 
dedicated multidisciplinary stroke team; and availability 
of case notes for review. 

Randomization 

We used a stratified-randomization approach. Hospitals 
were stratified by stroke performance (Sentinel Audit 
score above or below 60) in the 12 months preceding 
baseline data collection (2007 and 2008). Within each 
group, a computer-generated list was used to randomly 
allocate 12 hospitals to the intervention group and 12 to 



the control group. The intervention group took part in 
the QIC from January to October 2009, but the control 
group did not join the QIC until January 2010. The 
control group of hospitals, who waited to join the QIC, 
provided an untreated' control group. The nature of the 
trial meant that participants could not be blinded to 
group allocation. 

Sample size 

A conservative approach to the power calculation was 
undertaken, based on a simple baseline average versus 
end of study average comparison. This was preferable to 
basing the power calculation on an interrupted time 
series using a fixed number of charts per month, because 
we did not know in advance how the sites would choose 
to implement their activities or the likely impact of these 
activities (for example, in a simple linear fashion, stepped 
or incremental, or any other way). 

Various permutations of the power calculation were 
performed. Using data from the 2008 National Stroke 
Audit, the intra-class correlation coefficient was estimated 
at 0.149 for Bundle 1 and 0.217 for Bundle 2. Based on 
preliminary data, where at one hospital, compliance with 
Bundle 2 had improved to 80%, we assumed a 0.05 signifi- 
cance level, 90% power and a minimum detectable differ- 
ence of 25% for Bundle 1 and 35% for Bundle 2 in relative 
increased uptake between the control and intervention. 
We estimated that for improvement on Bundle 1 to reach 
significance, up to 24 hospitals, with 12 on either arm, 
would be required (based on the control group improving 
to 40% compliance and the intervention group improving 
to 65% compliance). For Bundle 2 to reach significance, 
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up to 19 hospitals (10 on either arm) would be required 
(based on the control group improving to 45% compliance 
and the intervention group improving to 70% compliance). 

Outcomes and follow-up 

Participating hospitals provided data to the QIC separately 
and independently from the National Audit (Table 2), 

Table 2 Baseline information for individual data (patient 
level) 



Intervention Control P 



Male 


2,287 


48.2% 


2,502 


47.3% 


0.559 


Comorbidities 












Atrial fibrillation 


714 


15.1% 


768 


14.5% 


0.457 


Previous stroke or TIA 


1,299 


27.4% 


1,404 


26.6% 


0.328 


Diabetes mellitus 


735 


15.5% 


742 


1 4.0% 


0.178 


Hyperlipidaemia 


647 


13.6% 


735 


13.9% 


0.942 


Hypertension 


2,087 


44.0% 


2,122 


40.1% 


0.135 


Myocardial infarction or angina 


777 


1 6.4% 


705 


13.3% 


0.019 


Valvular heart disease 


150 


3.2% 


118 


2.2% 


0.188 


Cardiac failure 


113 


2.4% 


135 


2.6% 


0.899 


Risk factors 












Smoker 


806 


1 7.0% 


765 


14.5% 


0.064 


Ex smoker 


733 


15.5% 


903 


17.1% 


0.453 


Alcohol excess 


275 


5.8% 


304 


5.7% 


0.692 


Patient independent pre stroke 


3,285 


69.3% 


4,031 


76.2% 


0.384 


Rp^pl mittpH within 7R H^x/^ 
i\caui i niicu win in i z.o uayo 


356 


7.5% 


308 


5.8% 


0.223 


rallclll UlcU a j II l|Jclllcl 11 


1 006 


21 2% 


1 01 1 


1 9 1 % 


0 1 89 


Ali\/p Hsx/Q 
Alive d L jU Udyo 


3 646 


76 9% 


3 834 


72 5% 


0 835 


Patient received antibiotics for a 
newly acquired pneumonia 
during admission after stroke 


298 


6.3% 


298 


5.6% 


0.729 


Patient had urinary tract infection 
in first seven days of admission 
defined by a positive culture or 
clinically treated 


159 


3.4% 


144 


2.7% 


0.514 


Motor deficits in first 24 hrs 
of admission 


2,534 


53.4% 


2,795 


52.9% 


0.727 


Dysphasia in first 24 hrs 
of admission 


1,588 


33.5% 


1,753 


33.2% 


0.748 


Dysarthria in first 24 hrs 
of admission 


1,549 


32.7% 


1,526 


28.9% 


0.808 


Motor deficits in first 24 hrs 
of admission 


2,534 


53.4% 


2,795 


52.9% 


0.879 


Patient worst level 
of consciousness 










0.992 


Unconscious 


527 


11.1% 


507 


9.6% 




Confused 


264 


5.6% 


248 


4.7% 




Drowsy 


657 


13.9% 


884 


16.7% 




Fully conscious 


2,933 


61.9% 


3,575 


67.6% 




Not known 


360 


7.6% 


73 


1 .4% 





although both were measuring the same processes. 
Random samples of 20 patients per month per hospital 
were used to generate data for both the intervention 
period (July 2008 to December 2009) and baseline pre- 
intervention period (July 2008 to December 2008). 

The primary measures for the trial were compliance 
with Bundle 1 and Bundle 2. The elements measured 
within those bundles were identical to the indicators 
used for the National Audit [8]. Bundle compliance was 
calculated using an all or none' measurement, where 
every process element of the bundle had to be under- 
taken for overall bundle compliance to be achieved [15]. 
Thus, omission of any one or more elements would 
result in a patients care being deemed 'bundle non- 
compliant' as illustrated in Figure 1. 

Data collection 

Once the QIC began in January 2009, intervention teams 
were asked to submit, every month, a complete registry 
of discharged patients coded for stroke from the previ- 
ous month (based on ICD 10 codes 61, 63, and 64). Each 
month, the Stroke 90:10 faculty used this registry to pro- 
duce the random sample of 20 patients for each hospital 
to audit. For each of the 20 patients included in the sam- 
ple, data were collected on 45 questions (excluding basic 
demographic information) on compliance with elements 
of Bundle 1 and Bundle 2, the three stages of acute 
stroke care: stroke onset and hospital stay; comorbidities 
and risk factors; and standards by discharge. Data were 
entered into a web-based database developed in partner- 
ship with the Royal College of Physicians. 

The QIC faculty collected the data for the baseline 
period for hospital teams in the intervention group, and 
throughout the entire study period for those in the con- 
trol group. After the baseline period the intervention 
group were expected to collect and submit data them- 
selves, though they sometimes required extensive faculty 
support to do so. 

In both the control and intervention groups, hospitals 
were asked to audit all patients in months where there 
were fewer than 20 admissions. Where patient records 
were incomplete, could not be retrieved, or were mis- 
coded, patients were excluded and replaced by the next 
available patient. In total, 1,328 (17%) records (control 
and intervention) were excluded (see Figure 2). 

Statistical analysis 

We used a difference-in-difference approach to compare 
the differences between the intervention and control 
groups on bundle compliance. This approach measures 
the difference in bundle compliance over time (before 
and after the intervention) for the intervention group 
compared with the difference over the same period for 
the control group. The analysis was carried out using 
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CT 


Aspirin 


Swallow 
Screen 


Weight 


All elements of 
the bundle 


Patient 1 


Y 


N 


Y 


N 


N 


Patient 2 


Y 


N 


N 


N 


N 


Patient 3 


Y 


Y 


Y 


Y 


Y 


Patient 4 


N 


Y 


N 


N 


N 


Patients 


Y 


N 


Y 


Y 


N 



Figure 1 Bundle compliance. In this example only patient 3 would be classed as receiving care that was bundle compliant. 



STATA (version 11, StataCorp LP, College Station, TX, 
USA). 

Using a pre-post-test methodology, we aimed to deter- 
mine whether there was a difference in relative average 
bundle compliance in the last three months of the base- 
line period (October 2008 to December 2008) compared 
with the last three months of the collaborative (October 
2009 to December 2009). Specifically, we sought to 
determine a difference in difference of means. We fitted 
a logistic regression model with a random effects term 
to take account of clustering at the hospital level. Bundle 
compliance was the primary measure. Explanatory 
variables included group (control or intervention), time 
period (October 2008 to December 2008 and October 
2009 to December 2009) and an interaction term between 



them. The interaction term provides the difference in dif- 
ferences that estimates the impact of the collaborative. 

Results 

Of the 25 eligible trusts in the Northwest of England, 24 
(covering 30 hospitals) agreed to participate and were 
randomized (Additional file 1). Of these, data were avail- 
able from 18 trusts for analysis (Figure 2), yielding 3533 
patients in the intervention arm and 3,059 patients in 
the control arm. Comparisons of patient demographic 
and clinical data between the control and intervention 
groups (Table 2) suggested no significant differences at 
baseline, except that the intervention group patients 
showed higher co-morbidity from myocardial infarction or 
angina (16.4% versus 13.3%; p = 0.019). The collaborative 



Assessed for eligibility (25 Hospital Trusts) 



Excluded 

Closed stroke services (1 Hospital Trust) 



Randomised (24 Hospital T'usts) 



Allocated to Intervention (12) 
Received allocated intervention (10) 

10 Trusts with 15 hospitals of which 5 were teaching; 
average stroke beds 27; strcke patierts per year 540 
Did not receive allocated intervention (2) 

Withdrawal by senior leaders after randomisation 



Lost to follow up: 0 

427 (11%) records not accessed (coding error 64, 
medical record unavailable 244, monthly admission 
below twenty 85, transfer 34). 
Discontinued Intervention (0) 



Clusters Analysed (9) 

9 Hospital Trusts, 3 teaching hospitals, average stroke 
beds 27, stroke patients per year 544 
Excluded from analysis (1 Hospital Trust) 

1 Tnist - submitted less than 50°o of required data 
Participants 

3533/3960 (89%) partic pants analysed 



Allocated to Control (12) 

Received allocated intervention (11) 

11 Trusts with 15 hospitals of which 6 were teaching; 
average stroke beds 24, stroke patierts per year 4 69 
Did not receive allocated intervention (1) 

1 hospital (merged with another organisation i 



Lost to follow up: 0 

901 (23%) records not accessed (coding error 34, medical 
record unavailable 723, monthly admission below twenty 
131, transfer 13). 

Discontinued Intervention (0.5) 

1 hospital (part of a Tru st comprising 2 hospitals) 

subsequently decided travel too much - withdrew M 



Clusters Analysed (9) 

9 hospital Trusts, 5 teaching hospitals, average stroke 

beds 27, stroke patients per year 483 

Excluded from analysis (1.5 Hospital Trusts) 

1.5 Trusts (1 Trust and the second hospital of the Trust 
that discontinued intervention) - submitted less than 50°o 
of requ ired data 
Parti ci pa nts 

3059/3960 (77%) participants analysed 



Figure 2 Flow diagram of participation. 
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program was run as designed. However, hospital sites did 
not consistently audit 20, or all, patients each month. 
Figure 2 shows that a small number of hospitals were 
excluded for having a reporting rate under 50%; this was 
pre-specified in the protocol. 

Primary measures 
Bundle 7 (early hours) 

Average Bundle 1 compliance in the control group at 
baseline (October 2008 to December 2008) was 24.3%, 
rising to 37.5% by study end (Figure 3). In the interven- 
tion group, compliance was 19.6% at baseline, rising to 
42.3% by study end. Using a difference-in-difference 
approach, this represents a relative increase in the odds 
of compliance of 1.56 (95% CI 1.06, 2.31 p = 0.025) in 
the intervention group compared with the control group 
(Table 3). These odds are equivalent to a relative im- 
provement by the end of the study of 10.9% (95% CI 
1.3%, 20.6%) in the intervention group relative to the 
control group. 

Secondary analysis found, within the bundle processes, 
that the largest relative difference was seen in administer- 
ing aspirin (or an alternative antiplatelet) within 24 hours 
of admission. The odds of aspirin (or suitable alternative) 
administration to patients where it was clinically indicated 
in the intervention relative to the control group increased 
by 1.52 (95% CI 1.05, 2.2 p = 0.027). 

Bundle 2 (rehabilitation bundle) 

Baseline bundle compliance with Bundle 2 in the control 
group was 21.9%, rising to 33.2% by study end. In the 
intervention group, baseline compliance was 27.3% and 
rose to 46.2% by study end (Figure 3). Using a difference- 



in-difference approach, this represents an increase in the 
odds of compliance of 1.61 (95% CI 1.07, 2.42 p = 0.023) 
in the intervention group compared with the control 
group. These odds are equivalent to a relative improve- 
ment by the end of the study of 11.2% (95% CI 1.4%, 
21.5%) in the intervention group relative to the control 
group (Table 3). 

Secondary analysis found that for the individual pro- 
cesses included in the bundle, both Mood Assessed during 
Hospital Admission (odds 1.74, 95% CI 1.17, 2.58; p = 
0.006) and Rehabilitation Goals set by multidisciplinary 
teams MDT (odds 2.43, 95% CI 1.7, 3.47; p < 0.001) in- 
creased in the intervention group more than in the con- 
trol group. 

Discussion 

Our study is among a small number of studies evaluating 
the effectiveness of a QIC that have used a controlled 
design. Like others [9,16], we have found evidence of im- 
provement in both the intervention and control groups 
pointing to an underlying secular trend of incremental im- 
provement over time. Some of the reasons for this are 
likely to lie with the unprecedented national and regional 
attention on stroke coinciding with the period of study. 
During this time, managed clinical networks for stroke 
were being developed, a National Audit Office report was 
published and the Department of Health Stroke Improve- 
ment program was launched [17]. Delivery of thromboly- 
sis was a national target, and the basic processes required 
for success in achieving this were closely linked to the 
processes of care packaged into our Early Hours bundle. 
Given the intensity of focus, it is unsurprising that both 
intervention and control groups demonstrated improved 




Control Group Intervention Group Control Group Intervention Group 

Figure 3 Bundle compliance over time in the control and intervention group. In bundle 1, for the control group, compliance went from 
24.3% to 37.5% (13.2% change) and in the intervention group, from 19.6% to 42.3% (22.7% change). On crude comparison, this is a difference of 
9.5%, but with all the various adjustments for clustering, this results in a relative benefit of 10.9% in the intervention relative to the control group. 

V . J 
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Table 3 The proportion of patients who received the bundle in the three months prior to the start of the collaborative 
intervention (October 2008 to December 2008) were compared with the proportion who received the bundle in the 
three months at end of the intervention (October 2009 to December 2009) 



Compliance: Proportion (%) Difference in differences of averages 





Control group 


Intervention group 










Oct-Dec 


Oct-Dec 


Oct-Dec 


Oct-Dec 


Odds ratio 95% CI 


P 


ICC* 




2008 


2009 


2008 


2009 


Percent relative improvement 






Early hours (Bundle 1) 


122/502 


212/566 


114/583 


258/610 


1.56 (1.06, 2.31) 


0.025 


0.066 




24.3% 


37.5% 


1 9.6% 


42.3% 


10.9% (1.3%, 20.6%) 






Brain Scan within 24 hrs of Hospital 


316/481 


435/556 


353/568 


455/603 


0.95 (0.65, 1 .4) 


0.799 


0.064 


AHmKc;inn 

/\U 1 1 1 loolUI 1 


65.7% 


78.2% 


62.1% 


75.5% 


-0.9% (-8.6%,5.4%) 






Aspirin in 24 hrs of hospital Admission 


151/392 


258/463 


115/436 


261/505 


1.44 (0.96, 2.15) 


0.079 


0.051 




38.5% 


55.7% 


26.4% 


51.7% 


9% (-196,18.8%) 






Swallowing Screening Recorded in 24 hrs 


220/422 


350/500 


326/495 


426/523 


1.09 (0.71, 1.67) 


0.7 


0.162 


of Hospital Admission 


52.1% 


70% 


65.9% 


81.5% 


2% (-8.4%, 11.2%) 






Weighed during Hospital Admission 


258/415 


346/504 


286/491 


462/525 


4.4 (2.75, 7.03) 


<0.001 


0.158 




62.2% 


68.7% 


58.2% 


88% 


26.2% (20.1%,30.6%) 






Rehabilitation (Bundle 2) 


110/502 
21.9% 


188/566 
33.2% 


159/583 
27.3% 


282/610 
46.2% 


1.61 (1.07, 2.42) 
11.2% (1.4%,21.5%) 


0.023 


0.197 


Ward of 50 percent + of stay 


325/502 


414/566 


384/583 


473/610 


1.17 (0.8, 1.72) 


0.412 


0.091 




64.7% 


73.1% 


65.9% 


77.5% 


3.0% (-4.6%,9.3%) 






Physiotherapist assessment in 72 hrs 


325/502 


414/566 


384/583 


473/610 


1 .6 (0.98, 2.6) 


0.06 


0.119 


of Hospital Admission 


64.7% 


73.1% 


65.9% 


77.5% 


9% (-0.4%, 16.1%) 






Occupational Therapist Assessment in 4 days 


302/406 


393/495 


394/490 


479/534 


1.06 (0.68, 1.67) 


0.789 


0.234 


of Hospital Admission 


74.4% 


79.4% 


80.4% 


89.7% 


1.4% (-9.5%, 11.1%) 






Mood Assessed during Hospital Admission 


235/388 


354/478 


330/469 


419/515 


2.68 (1.69, 4.26) 


<0.001 


0.247 




60.6% 


74.1% 


70.4% 


81.4% 


24.1% (12.9%,34%) 






Rehabilitation Goals set during Hospital 


147/407 


226/488 


192/478 


315/512 


5.43 (3.26, 9.05) 


<0.001 


0.194 


Admission agreed by MDT 


36.1% 


46.3% 


40.2% 


61.5% 


35.8% (27.396,41.9%) 







*/CC: The Intraclass Correlation Coefficient indicates the variation due to patients being clustered within hospitals. The ICC estimates shown here are derived from 
the associated random effects model. 

In line one the numerator is the number of patients who received all four process measures; the denominator is the total number surveyed in the three 
month period. 



performance. The trends we observed are significant and 
signal the collective impact of direction setting (through 
national policy), implementation of national standards, 
regulation, clinical leadership, research implementation 
and other activities in creating a rising tide' of more reli- 
able care for stroke patients. 

Participation in a QIC does appear to offer a modest 
overall effect in the amplitude of change over and above 
secular trend. At first glance it may seem disappointing 
that the overall benefits are not more significant. How- 
ever, the bundled nature of our primary outcome mea- 
sures may have obscured some important and significant 
effects. These include the disproportionate impact of 
Stroke 90:10 on some process indicators of quality of 
care and the resistant nature of others. Looking at the 
bundles of care as individual items, it is possible to see 
that four care processes (administration of aspirin/ 



antiplatelet, weighing patients on time, mood assessment 
and setting MDT rehabilitation goals) improved signifi- 
cantly in the group of hospitals that participated in the 
collaborative. Weighing patients and setting MDT goals 
conferred a 17% and 21% absolute improvement respect- 
ively. One possibility, therefore, is that the effects of 
QICs may be specific rather than generalized: some pro- 
cesses of care may be more susceptible to the improve- 
ment methods used in a collaborative program than 
others. This is an important new hypothesis that requires 
further testing. Furthermore, it appears that these pro- 
cesses could have some distinctive characteristics: they are 
(relatively) simple changes under the direct control of the 
hospital stroke team on the stroke unit (defined as a phys- 
ical unit and not a collection of staff and services) and 
they are geographically bounded. This possible explan- 
ation is supported somewhat by an Australian study in 
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acute stroke units, which showed that focused improve- 
ments in direct patient care, delivered on a ward, can be 
made using a collaborative method [18]. 

Processes of care that are less tractable may require 
more attention to engineering systems and processes. For 
example, delivering a CT scan within hours of admission 
requires coordination of multiple services and provider 
groups. Achieving the necessary changes in patterns of 
working, culture, and attitudes requires an approach dif- 
ferent from that used for less socially complex problems. 
Our study may thus provide some evidence to suggest that 
the effects of QICs, rather than being uniform, may be 
mediated by the nature of the processes being targeted 
and the amount of control participants have over them. 
Collaboratives may be more suitable for some kinds of im- 
provement interventions than others, and future research 
should investigate this possibility further. The kinds of im- 
plementation interventions needed to support more com- 
plex organizational change also require further study, now 
that promising examples are beginning to emerge [19-24]. 

Limitations 

Though our study design represents improvement on 
the weaker designs typically used in evaluations of col- 
laboratives, it does have some limitations. First, because 
the nine processes used in the study were those used by 
the National Audit, we may not have captured other 
processes of care that are clinically more important. 
Variation in data collection approaches across hospitals, 
although small, may have introduced bias to the results. 
Second, six hospitals did not provide data, and there was 
some variation in completeness rates between hospitals. 
This may introduce bias because it is unlikely the hospi- 
tals that failed to provide data, or did not produce 
complete data, were similar to those that did. Third, 
generalizability beyond an English context is limited. 
The socio-technical nature of this intervention, and the 
context within which it was set, suggests that it is not 
clear whether the same effects would be found in other 
populations or settings. 

Fourth, our study was not designed to explain whether 
the increased process adherence resulted in improved out- 
comes or had any unintended consequences, for example 
longer lengths of stay or increased re-admissions to hos- 
pital. Future studies should investigate the possibility that 
there may be some causal mechanism that links improved 
performance on the nine processes to beneficial or un- 
wanted outcomes. Finally, beyond evaluations of collabo- 
ratives, experimental evaluations of many complex socio- 
technical interventions demonstrate modest effect sizes, 
which might suggest the limitations of positivist evalua- 
tions in this context and signal a requirement for more so- 
phisticated evaluation approaches. Some of these issues 
are the subject of an accompanying paper [25]. 



Conclusions 

Our study suggests that the answer to whether a Break- 
through Series QIC can deliver the extra boost needed to 
induce improvement beyond secular trend is not straight- 
forward. It does appear to support improvement in more 
consistent delivery of some processes of care grouped into 
bundles, but additional, or other kinds of, support may be 
needed for more complex organizational challenges. Our 
study reinforces the need, when researching health service 
improvements, for controlled studies using difference-in- 
difference analyses to avoid mistaking secular trends for 
treatment effects. Delivering consistently high quality of 
stroke care remains a key challenge. 
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